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Quick Overview Nelle’ 


University 


= Getting past Automated Vehicle (AV) safety rhetoric 
= AV safety in a nutshell 
= Policy points: 
e Societal benefits 
e Public road testing 
e Municipal preemption 
e SAE Level 2/2+/3 issues 
e Federal vs. state regulation 
e Other policy issues 
= Revisiting common myths 
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Instead, you get Robot Error 


Two Cruise cars in San Francisco became wrapped in downed Muni wires and 
caution tape at Leavenworth Street and Clay Street on March 21, 2022. 


Courtesy of John-Phillip Bettencourt 
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Getting Past the AV Safety Rhetoric Mellon 


University 


= Nobody knows when/if Autonomous Vehicles (AVs) 
will be safer than human drivers 
e Reduced fatality rates are aspirational 
= Some humans drive drunk 
e On average they are still good and adaptable 
= But computers lack common sense 
e Machine Learning can fail in novel situations 


= Computer drivers can be imperfect 
e Might hit a bus — even if lidar sees the bus 
e Safety must be engineered, not assumed 


© 2023 Philip Koopman 4 


Elements of AV Safety Mito 

1. Safe as a human driver on average 

e ~75M to 125M miles/fatality for S.F., including impaired 
2. Avoiding risk transfer onto vulnerable populations 

e Pedestrian harm should not increase — even if net harm is reduced 
3. Avoid negligent computer driving 

e Running red lights and stop signs is not OK 
4. Conform to industry safety standards 

e Including SAE J3018 for public road testing 
5. Address other ethical & equity concerns 

e Avoid local preemption; manufacturer accountability for harm 
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Policy Point: Societal Benefit Mellon 
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= Benefits accrue only after AVs are safe and reliable 
e 100++ million miles to confirm reduced fatalities 
e Near term, “safe” might mean lower reliability 


A Swarm of Self-Driving Cruise 


> Ask the hard questions Taxis Blocked San Francisco 


Traffic for Hours 


e What benefits will there be right now? 
— “Benefits disabled” but no wheelchair access 
— “Already saving lives” with about 1 million miles WF 


— 


- “Promise unprofitable thing X” with no regulation |). 234). .eaaets 
: 4 A . wl as, i= 2 a : ) . 
e What public costs will there be right now? 5 ih ; 
— Congestion and blocked emergency responders ; https://bit.ly/45xmpdo 


— Risk of harm from still-under-development software on public roads 
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= There is no such thing as driverless testing 


> Safety driver should stay in 
until safety is proven 
e Require SAE J3018 testing safety standard 
e Test with driver not touching controls 
e ‘Beta’ is road testing, not SAE Level 2 


> Driver-out should be testing the business model, not atau 
e Software updates need driver-in qualification testing 


© 2023 Philip Koopman 7 


Carnegie 


Policy Point: Municipal Preemption Mellon 


University 


= Companies push for municipal preemption 
e Argue that cities will ban AV testing Two Wave Cars Block San Francisco 
e How is this working out in San Francisco? ii ae 


Incidents Rise 300 Percent 


The self-driving cars were left motionless by heavy San Francisco traffic due to the 
Pride Parade and Giants game on Sunday. 


> Middle ground: responsive to local conditions = 

e Munis can forbid testing for specific situations 
— School zones during student transit times ay 
— Parades, fires, 1st amendment events, construction ar 

e Munis can selectively suspend service 
— Keep away from emergency scenes after firefighter incidents 
— Avoid high-activity neighborhoods after fire truck crash 

e Munis must be able to enforce traffic laws 


00 ° © © ts: //bit.ly/3DZTpza 
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Policy Point: SAE Level 2/2+/3 Vehicles Mellon 


University 


= Already deployed on roads 
e Fatalities, injuries due to driver complacency 
e No substantive regulations beyond sparse recalls 
= Level 3 “gives driver time back” 
e Is driver criminally liable for a fatality? 
e Mercedes Benz only talks about product liability 
> Create a clear duty of responsibility for the computer driver 
e When computer is driving, manufacturer is responsible party 
e Defined, non-zero safe harbor transition time back to human driver 
e Liability attaches to manufacturer for inadequate driver monitoring 
> Detailed proposal for state regulation on this topic — o2xv2sphiipxoopman 9 
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Policy Point: Federal vs. State Regulation Mellon 
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= Problem: computer driver is “equipment” 


> NHTSA/FMCSA should control equipment 
e Ability of computer driver to adhere to state laws 
e ANPRM NHTSA-2020-0106 AV framework 


> States should control computer driver behavior & 
e Hold computer drivers to same duty of care as a human driver 
e Determine and enforce driving behavioral rules 
e Ability to revoke driver licenses based on negligent driving 
e Munis need ability to enforce & report negligence to state DMV 
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Other Policy Points Mellon 


University 


= Economic impact 
e Saturation of fully automated vehicles is decades away 
e City-by-city / route-by-route for foreseeable future 
e Still need humans for remote support, maintenance, testing 
e Still need people for last few feet of delivery, security, etc. 
= Required arbitration 
e Impairs transparency (and safety) with secret outcomes 
e Degrades balance of power if mandatory 
m= Equity issues 
e Will underserved populations benefit without regulatory mandate? 
e Will road testing impose risk on the vulnerable? 93053 Phila Kooper 4 
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__Quick List of Myths @ EE 
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= “Humans are terrible drivers” / “94% Human Error” 
e Computers lack common sense; they make mistakes too 
m= “We have 5 MILLION miles of testing” 
e Proof of saving lives requires 100 million to 1 billion miles 
m “Level 2/2+ makes cars safer” 
e AEB helps safety; Some Level 2/2+ systems decrease safety 
m “We, the manufacturer, take responsibility” (for product liability) 
e The more urgent issue is tort/criminal, not product liability 
= “Current laws and regulations are enough’ = FALSE 
e Liability issues; Software safety regulation; L2+ loophole 
e Little regulatory pressure for promised benefits © 2023 Philip Koopman 42 
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Resources Mellon 
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m Liability-based proposal for state AV regulation & podcast 


e https://safeautonomy.blogspot.com/2023/05/a-liability-approach-for- 
automated.html 


m Video lecture series on autonomous vehicle safety: 
e Keynote AV Safety overview video : https://youtu.be/oE_2rBxNrfc 
e Mini-course: https://users.ece.cmu.edu/~koopman/lectures/index.html#av 
m “Safe Enough” book & talk video: 
e https://safeautonomy.blogspot.com/2022/09/book-how-safe-is-safe-enough- 
measuring.html 
m UL 4600 AV safety standard book & talk video: 
e https://safeautonomy.blogspot.com/2022/11/blog-post.html 
m US House E&C testimony: 
e https://safeautonomy.blogspot.com/2023/07/av-safety-claims-and-more-on-my.html 
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